Hybrid Feature Selection Approach Research Articles

AI-enabled prediction of sim racing performance using telemetry data

Despite the emerging and rapid progress of esports, approaches for ensuring high-quality analytics and training among professional and amateur esports teams are lacking. In this paper, we demonstrate how the application of data science techniques and Machine Learning (ML) approaches in esports, particularly in sim racing science, can illuminate the most important in-game metrics that dictate performance. Thus, using a professional racing simulator and MoTec i2 Pro (v1.1.5, Australia), we gathered extensive telemetry data from 174 participants, who completed 1327 laps on the Brands-Hatch circuit in the Assetto Corsa Competizione (v1.9, KUNOS Simulazioni). We clustered the obtained laps based on performance (lap-time), and then identified driving behaviors within performance groups. We also analyzed the feature subset obtained from a hybrid feature selection approach using two correlation analyses and three ML models.The best model achieved a prediction accuracy of 97.19%, demonstrating that the model effectively captured the critical factors that influenced driving performance during a lap. The results confirm that average speed is the most important metric, followed by lateral acceleration, steering angle, and lane deviation. Our analyses offer key metrics for refining training tools and techniques in sim racing performance improvement.

Empirical Enhancement of Intrusion Detection Systems: A Comprehensive Approach with Genetic Algorithm-based Hyperparameter Tuning and Hybrid Feature Selection

AbstractMachine learning-based IDSs have demonstrated promising outcomes in identifying and mitigating security threats within IoT networks. However, the efficacy of such systems is contingent on various hyperparameters, necessitating optimization to elevate their performance. This paper introduces a comprehensive empirical and quantitative exploration aimed at enhancing intrusion detection systems (IDSs). The study capitalizes on a genetic algorithm-based hyperparameter tuning mechanism and a pioneering hybrid feature selection approach to systematically investigate incremental performance improvements in IDS. Specifically, our work proposes a machine learning-based IDS approach tailored for detecting attacks in IoT environments. To achieve this, we introduce a hybrid feature selection method designed to identify the most salient features for the task. Additionally, we employed the genetic algorithm (GA) to fine-tune hyperparameters of multiple machine learning models, ensuring their accuracy in detecting attacks. We commence by evaluating the default hyperparameters of these models on the CICIDS2017 dataset, followed by rigorous testing of the same algorithms post-optimization through GA. Through a series of experiments, we scrutinize the impact of combining feature selection methods with hyperparameter tuning approaches. The outcomes unequivocally demonstrate the potential of hyperparameter optimization in enhancing the accuracy and efficiency of machine learning-based IDS systems for IoT networks. The empirical nature of our research method provides a meticulous analysis of the efficacy of the proposed techniques through systematic experimentation and quantitative evaluation. Consolidated in a unified manner, the results underscore the step-by-step enhancement of IDS performance, especially in terms of detection time, substantiating the efficacy of our approach in real-world scenarios.

Early diagnosis of obsessives-compulsive disorder through gene expression analysis using machine learning models

The primary aim of this research study is to develop an early diagnosis method for obsessive-compulsive disorder (OCD) by utilizing gene expression analysis and Machine Learning techniques. Gene expression data from both blood and brain samples were collected from the gene expression omnibus (GEO) database. As OCD cannot currently be detected through instruments, it relies on clinical symptoms that are often misinterpreted. To address this, a novel hybrid feature selection approach that combines statistical and ML methods is proposed to identify down-regulated genes that may play a crucial role in the development of OCD. The results of the earlier studies point to important implications and emphasize the significance of down-regulated gene expression in OCD. Currently, gene expression profiling is used as an investigative tool to identify the specific cell receptors associated with certain conditions, followed by targeted medication to alleviate symptoms. Our proposed method achieved high accuracy rates of 83% for blood data and 92% for brain data when compared to other feature selection methods such as MIFS, CFS, and mRMR, using various Machine Learning models. These results demonstrate the effectiveness of our approach in early OCD diagnosis using gene expression analysis. https://github.com/Naseerullah-Qureshi/classify_gene_expression/blob/main/gene_expression.txt.

Erratum to: A hybrid feature selection approach for urinary tract infection detection and prediction in IoT-fog environment

The online version of the original article can be found at https://doi.org/10.31893/multiscience.2024080

Accelerated prediction of stacking fault energy in FCC medium entropy alloys using multilayer perceptron neural networks: correlation and feature analysis

A multilayer perceptron neural networks (MLPNN) model is developed for robust and quick prediction of stacking fault energy (SFE) to overcome the challenges faced in the calculation of SFE via experimentation and atomistic calculations in FCC medium entropy alloys (MEA). The present investigation employs a three-step hybrid feature selection approach to obtain a comprehensive understanding of the prominent features that influence the SFE, as well as the interrelationships among these features. The feature space encompasses various features related to composition, lattice stability, and elemental properties, of MEAs. The findings indicate that the estimation of SFE relies on five crucial factors: temperature, lattice stability, specific heat, ionization energy, and Allen electronegativities. Furthermore, a mathematical relationship for the estimation of the SFE is derived, considering the various influencing and prominent factors. Consequently, the MLPNN model for robust SFE prediction in MEAs is developed and the performance is evaluated using R 2 scores, with values of 0.87 and 0.85 obtained for the training and testing datasets, respectively. This efficient strategy introduces a novel opportunity for the engineering of SFE in the extensive range of alloy chemistry of MEAs, enabling the quick prediction of SFE, and facilitating the systematic exploration of new alloys for the development of mechanisms that may accommodate deformation through octahedral/partial slip, twinning, and/or transformation.

Bi-stage feature selection for crop mapping using grey wolf metaheuristic optimization

Machine learning classifiers have been widely used for crop mapping; however, employing numerous features can reduce classifier accuracy. Feature selection is a widely recognized technique that addresses the issue of high dimensionality by choosing the most relevant subset of features with the highest degree of relevance and practicality. Therefore, the optimal combination of feature selection methods and classifiers for high-precision crop mapping remains an open problem. In this study, we introduce a novel hybrid feature selection approach to obtain the optimal subset of features for effective crop mapping. Initially, spectro-temporal remote sensing features from the dataset were ranked using two distinct feature selection techniques: Mutual Information (MI) and ReliefF filter. The most relevant features from each filter-based approach were combined into a union subset. Subsequently, the Gray Wolf Optimization (GWO), a metaheuristic optimization technique, was applied to enhance the feature set generated in the initial step. In the final stage, a random forest classifier leveraged the optimized feature subset for accurate crop type prediction. The effectiveness of the proposed approach was evaluated in Behiera governorate, Egypt. Comparative analysis against existing crop mapping methods demonstrated the superior performance of the proposed approach, achieving an accuracy rate of 82%. Furthermore, this approach showcased a significant reduction in the feature count, streamlining the set from 308 down to only 26 features.

Prediction of futile recanalisation after endovascular treatment in acute ischaemic stroke: development and validation of a hybrid machine learning model

BackgroundIdentification of futile recanalisation following endovascular therapy (EVT) in patients with acute ischaemic stroke is both crucial and challenging. Here, we present a novel risk stratification system based on hybrid...

A HYBRID FEATURE SELECTION APPROACH FOR ROMAN URDU TEXT CLASSIFICATION

Text classification is the task of assigning labels to unlabeled text data. Text classification has several applications like sentiment analysis, document classification, and fake news detection such as Machine learning (ML) methods have been used commonly in text classification in the last several years. The fundamental problem in ML is that these approaches heavily depend on feature selection methods. The models and feature selection methods used in this research. Several past types of research conclude that there is no uniform feature selection method that works well for all types of classifier tasks as well as Urdu is a resource-poor language. In this study, a proposed hybrid feature selection approach for Roman Urdu text not only reduces the dimension of the feature map but also increases the accuracy of ML models. Using 11000 and 20000 records have been used for Support Vector Classifier, Naive Base and Decision Tree which have given 80.81%, 72.94% and 76.78% respectively, among other tested methods. The best accuracy values achieved by each classifier and the hybrid features ChiSAE, CorrelationAE, and GainRAE. In future, text classification for better understanding of human being self-analysis as well as deep learning methods will be utilized for better authenticity.

A Hybrid Feature Selection Approach Incorporating Mutual Information and Genetics Algorithm for Web Server Attack Detection

A Hybrid Feature Selection Approach Incorporating Mutual Information and Genetics Algorithm for Web Server Attack Detection

A hybrid feature selection approach for urinary tract infection detection and prediction in IoT-Fog environment

Urinary Tract Infections (UTIs) are a prevalent health concern experienced by millions of individuals worldwide and have a significant impact on overall well-being. The clinical presentation of UTIs varies depending on their location and severity. Common symptoms include discomfort during urination, increased frequency, a compelling urge to urinate, discomfort in the lower abdomen, and the presence of blood in the urine (hematuria). Severe cases may be accompanied by fever, pain in the flank region, and other systemic symptoms, indicative of an upper UTI. Precise diagnosis of UTIs is achieved through clinical evaluation and laboratory tests. The primary aim of this research is to utilize appropriate Machine Learning (ML)-based algorithms to predict Urinary Tract Infections (UTIs). The ultimate goal is to develop a predictive model that can be effectively implemented in a smart toilet system. By achieving this objective, this research seeks to provide an innovative and pragmatic solution for UTI prediction, harnessing the potential of ML algorithms in IoT-Fog settings to enhance healthcare and promote public health. This paper introduces a hybrid approach that combines feature selection methods and employs the Guided Regularized Random Forest (GRRF) classification algorithm to aid in the diagnosis of UTIs. A UTI dataset was created using data from routine examinations and definitive diagnostic outcomes for UTI patients. Dimensionality reduction was carried out using Principal Component Analysis (PCA), while feature selection was performed using K-best and Lasso CV techniques. Through the proposed strategy, this study achieved a remarkable accuracy rate of 98.8% and a precision rate of 98.90% in UTI identification. Future UTI prevention and treatment plans must be optimized via further research and ongoing efforts to overcome antibiotic resistance.

Elastic net-based high dimensional data selection for regression

High-dimensional feature selection is of particular interest to researchers. In some domains, such as microarray data, it is quite common for a group of highly correlated explanatory variables to be of equal importance for inclusion in the predictive model. This paper proposes a new hybrid feature selection approach that integrates feature screening based on Kendall’s tau and Elastic Net regularized regression (K-EN). K-EN as an approach that embeds the Elastic Net, has the advantage of the grouping effect, which automatically includes all the highly correlated variables in the group. The K-EN approach offers insightful solutions to high-dimensional regression problems and improves Elastic Net performance since the screening phase is preceded by a step that further reduces the number of explanatory variables by removing those that disagree with the target based on Kendall’s tau. The use of Kendall’s tau further enhances Elastic Net performance, as it is robust enough to handle heavy-tailed distributions, non-parametric models, outliers, and non-normal data with greater ease. K-EN is therefore a time-saving approach. The proposed algorithm is evaluated on four simulation scenarios and four publicly available datasets, including riboflavin, eyedata, Longley, and Boston Housing, and achieves 0.2528, 0.0098, 0.1007, and 0.4121 respectively as the Mean Squared Error (MSE). K-EN’s MSEs are the best compared to those achieved by the state-of-the-art approaches reviewed in this paper. In addition, K-EN selects up to 100% of relevant features when run on simulated data.

A Multivariate Time Series Analysis of Electrical Load Forecasting Based on a Hybrid Feature Selection Approach and Explainable Deep Learning

In the smart grid paradigm, precise electrical load forecasting (ELF) offers significant advantages for enhancing grid reliability and informing energy planning decisions. Specifically, mid-term ELF is a key priority for power system planning and operation. Although statistical methods were primarily used because ELF is a time series problem, deep learning (DL)-based forecasting approaches are more commonly employed and successful in achieving precise predictions. However, these DL-based techniques, known as black box models, lack interpretability. When interpreting the DL model, employing explainable artificial intelligence (XAI) yields significant advantages by extracting meaningful information from the DL model outputs and the causal relationships among various factors. On the contrary, precise load forecasting necessitates employing feature engineering to identify pertinent input features and determine optimal time lags. This research study strives to accomplish a mid-term forecast of ELF study load utilizing aggregated electrical load consumption data, while considering the aforementioned critical aspects. A hybrid framework for feature selection and extraction is proposed for electric load forecasting. Technical term abbreviations are explained upon first use. The feature selection phase employs a combination of filter, Pearson correlation (PC), embedded random forest regressor (RFR) and decision tree regressor (DTR) methods to determine the correlation and significance of each feature. In the feature extraction phase, we utilized a wrapper-based technique called recursive feature elimination cross-validation (RFECV) to eliminate redundant features. Multi-step-ahead time series forecasting is conducted utilizing three distinct long-short term memory (LSTM) models: basic LSTM, bi-directional LSTM (Bi-LSTM) and attention-based LSTM models to accurately predict electrical load consumption thirty days in advance. Through numerous studies, a reduction in forecasting errors of nearly 50% has been attained. Additionally, the local interpretable model-agnostic explanations (LIME) methodology, which is an explainable artificial intelligence (XAI) technique, is utilized for explaining the mid-term ELF model. As far as the authors are aware, XAI has not yet been implemented in mid-term aggregated energy forecasting studies utilizing the ELF method. Quantitative and detailed evaluations have been conducted, with the experimental results indicating that this comprehensive approach is entirely successful in forecasting multivariate mid-term loads.

A novel hybrid feature selection and ensemble-based machine learning approach for botnet detection

In the age of sophisticated cyber threats, botnet detection remains a crucial yet complex security challenge. Existing detection systems are continually outmaneuvered by the relentless advancement of botnet strategies, necessitating a more dynamic and proactive approach. Our research introduces a ground-breaking solution to the persistent botnet problem through a strategic amalgamation of Hybrid Feature Selection methods—Categorical Analysis, Mutual Information, and Principal Component Analysis—and a robust ensemble of machine learning techniques. We uniquely combine these feature selection tools to refine the input space, enhancing the detection capabilities of the ensemble learners. Extra Trees, as the ensemble technique of choice, exhibits exemplary performance, culminating in a near-perfect 99.99% accuracy rate in botnet classification across varied datasets. Our model not only surpasses previous benchmarks but also demonstrates exceptional adaptability to new botnet phenomena, ensuring persistent accuracy in a landscape of evolving threats. Detailed comparative analyses manifest our model's superiority, consistently achieving over 99% True Positive Rates and an unprecedented False Positive Rate close to 0.00%, thereby setting a new precedent for reliability in botnet detection. This research signifies a transformative step in cybersecurity, offering unprecedented precision and resilience against botnet infiltrations, and providing an indispensable blueprint for the development of next-generation security frameworks.

Breast Cancer Detection and Classification Using Hybrid Feature Selection and DenseXtNet Approach

Breast Cancer (BC) detection and classification are critical tasks in medical diagnostics. The lives of patients can be greatly enhanced by the precise and early detection of BC. This study suggests a novel approach for detecting BC that combines deep learning models and sophisticated image processing techniques to address those shortcomings. The BC dataset was pre-processed using histogram equalization and adaptive filtering. Data augmentation was performed using cycle-consistent GANs (CycleGANs). Handcrafted features like Haralick features, Gabor filters, contour-based features, and morphological features were extracted, along with features from deep learning architecture VGG16. Then, we employed a hybrid optimization model, combining the Sparrow Search Algorithm (SSA) and Red Deer Algorithm (RDA), called Hybrid Red Deer with Sparrow optimization (HRDSO), to select the most informative subset of features. For detecting BC, we proposed a new DenseXtNet architecture by combining DenseNet and optimized ResNeXt, which is optimized using the hybrid optimization model HRDSO. The proposed model was evaluated using various performance metrics and compared with existing methods, demonstrating that its accuracy is 97.58% in BC detection. MATLAB was utilized for implementation and evaluation purposes.

AER-HYBRITECH: Averaging Ensemble Regression with Hybrid Encoding and Enhanced Feature Selection Technique for Predictive Maintenance

Predictive maintenance is critical to modern industrial operations, preventing unexpected equipment failures and minimizing downtime. Existing methods often encounter challenges related to data preprocessing, missing data imputation, and feature selection. This paper presents "AER-HYBRITECH," a novel approach that addresses these challenges and enhances the predictive maintenance process. Traditional methods overlook the intricate relationships within the data, resulting in suboptimal predictive performance. To bridge this gap, the proposed AER-HYBRITECH algorithm is introduced. AER-HYBRITECH stands out in several ways. Firstly, it utilizes a hybrid encoding technique that converts categorical data into a more informative numerical representation by incorporating the average values of label-encoded data and its frequency, leading to improved feature utilization. Furthermore, it introduces the AER-MDI (Averaging Ensemble Regression-based Missing Data Imputation) technique, which combines M5P, REPTree, and linear regression models to impute missing data, ensuring a more complete dataset. The algorithm also implements Min-Max normalization to scale numeric features, making them compatible for further analysis. One of the key innovations of AER-HYBRITECH is its enhanced hybrid feature selection (EHFS) approach. The AER-HYBRITECH algorithm transforms and preprocesses the data and ensures that predictive maintenance models are built on a solid foundation, resulting in more accurate predictions and reduced maintenance costs.

Feature selection in intrusion detection systems: a new hybrid fusion of Bat algorithm and Residue Number System

ABSTRACT This research introduces innovative approaches to enhance intrusion detection systems (IDSs) by addressing critical challenges in existing methods. Various machine-learning techniques, including nature-inspired metaheuristics, Bayesian algorithms, and swarm intelligence, have been proposed in the past for attribute selection and IDS performance improvement. However, these methods have often fallen short in terms of detection accuracy, detection rate, precision, and F-score. To tackle these issues, the paper presents a novel hybrid feature selection approach combining the Bat metaheuristic algorithm with the Residue Number System (RNS). Initially, the Bat algorithm is utilized to partition training data and eliminate irrelevant attributes. Recognizing the Bat algorithm's slower training and testing times, RNS is incorporated to enhance processing speed. Additionally, principal component analysis (PCA) is employed for feature extraction. In a second phase, RNS is excluded for feature selection, allowing the Bat algorithm to perform this task while PCA handles feature extraction. Subsequently, classification is conducted using naive bayes, and k-Nearest Neighbors. Experimental results demonstrate the remarkable effectiveness of combining RNS with the Bat algorithm, achieving outstanding detection rates, accuracy, and F-scores. Notably, the fusion approach doubles processing speed. The findings are further validated through benchmarking against existing intrusion detection methods, establishing their competitiveness.

Ensemble classifiers with hybrid feature selection approach for diagnosis of coronary artery disease

Coronary artery disease (CAD) is a common type of cardiovascular disease with a high mortality rate worldwide. As symptoms may not be recognized until, after the cardiac attack, early diagnosis and treatment are critical to lowering mortality. The proposed study focuses on the creation of an intelligent ensemble system for the accurate detection of CAD. This paper presents the hybrid feature selection method based on Lasso, random forest-based boruta, and recursive feature elimination methods. The significance of a feature is determined by the score each approach provides. Machine learning techniques such as random forest, support vector machine, K-nearest neighbor, logistic regression, decision tree, and Naive Bayes are developed as base classifiers. Then, ensemble techniques like bagging and boosting models are created using base classifiers. The Z-Alizadeh Sani dataset was used to build and test the model. The bagged random forest model achieved 97.6% accuracy and 100% recall. The CatBoost model achieved 97.7% accuracy and 99.0% recall. Compared to traditional classifiers, the ensemble models achieved higher accuracy and can be used to assist clinicians in diagnosing coronary artery disease

A Hybrid Feature Selection Approach based on Random Forest and Particle Swarm Optimization for IoT Network Traffic Analysis

The complexity and volume of network traffic has increased significantly due to the emergence of the “Internet of Things” (IoT). The classification accuracy of the network traffic is dependent on the most pertinent features. In this paper, we present a hybrid feature selection method that takes into account the optimization of Particle Swarms (PSO) and Random Forests. The data collected by the security firm, CIC-IDS2017, contains a large number of attacks and traffic instances. To improve the classification accuracy, we use the framework's RF algorithm to identify the most important features. Then, the PSO algorithm is used to refine the selection process. According to our experiments, the proposed method performed better than the other methods when it comes to the classification accuracy. It achieves a ~99.9% accuracy when using a hybrid of Random Forest and PSO. The hybrid approach also helps improve the model's performance. The suggested method can be utilized by security analysts and network administrators to identify and prevent attacks on the IoT.

A deep learning approach for the depression detection of social media data with hybrid feature selection and attention mechanism

AbstractDepression is a severe mental health issue. The user‐generated content on social media (SM) is growing nowadays. Some computational approaches have been proposed for detecting depression based on users' SM data. However, because of the use of formal language, short range of words and misspellings in the SM data, depression detection (DD) is a challenging task. This paper proposes a novel deep learning (DL) technique for performing DD of the SM data with the help of the hybrid feature selection (FS) mechanism. Initially, two publicly available datasets containing user tweets are collected for implementing the proposed research model. Then the collected datasets are preprocessed for further processing. The preprocessing phase includes critical processes that contribute to creating a ready‐to‐use dataset for training and testing. After preprocessing, the preprocessed data is divided into prime and non‐prime words based on the dictionary approach. After that, the hybrid FS approach is implemented to select the most relevant features from the prime and non‐prime words for higher classification accuracy (AC). In the hybrid model, firstly Term Frequency Inverse Document Frequency integrated Modified Information Gain (TFIDF‐MIG) approach is proposed that assigns the score value of each prime and non‐prime word in the dataset. Secondly, optimal features are selected from the weighted features using the Improved Elephant Herding Algorithm (IEHA). Finally, the decided features from the hybrid model are fed into the DL model, namely attention included improved ReLU‐based Convolution Neural Network with Long Short‐Term Memory (AIRCNN‐LSTM) for DD. Experiments are performed on the collected datasets to assess the proposed model's performance efficiency. The results of the extensive experiments show that the presented work outperforms existing techniques regarding DD classification AC by locating the best solutions. At the same time, it reduces the number of features chosen.

Computational model for breast cancer diagnosis using HFSE framework

Mammography is one of the imaging modalities used in diagnosing breast cancer at an earlier stage. Misdiagnosis leads to risks for the patients. Better feature extraction and selection techniques can reduce misdiagnoses as they are essential in better-performing classifiers. The proposed approach in this paper introduces a novel Hybrid Feature Extraction and Hybrid Feature Selection (HFSE) framework. It uses mammograms from Digital Database for Screening Mammography (DDSM) datasets to classify mammograms into benign and malignant images. This paper presents a novel hybrid feature extraction approach from the Gray level co-occurrence matrix (GLCM), Linear local binary pattern (LBP), Gabor, and Tamura. Combinedly, it extracts 104 features to train the advanced classifiers such as Logistic Regression, Linear Perceptron, Support Vector Machine, Decision Tree, and Artificial Neural Network. The proposed Hybrid feature extraction method vigorously compares combinations of existing single feature extraction methods. The paper also presents a novel hybrid feature selection approach to choose a subset of the most relevant feature. It compares the Intrinsic feature with the proposed hybrid feature selection method. Hyperparameter tuning and Pipeline optimization techniques applied to the classifiers improve their performance metrics. The experimental results show that the proposed framework performs better using Hybrid feature extraction and feature selection on Artificial Neural Networks. This paper makes a comparative analysis of the related works. It outperforms by achieving a classifier accuracy of 94.57%, specificity of 91.80%, the sensitivity of 95.59 %, and an F1-score of 94.89% on Artificial Neural Networks.